Search results for "bregman divergence"
showing 7 items of 7 documents
Blind deconvolution using TV regularization and Bregman iteration
2005
In this paper we formulate a new time dependent model for blind deconvolution based on a constrained variational model that uses the sum of the total variation norms of the signal and the kernel as a regularizing functional. We incorporate mass conservation and the nonnegativity of the kernel and the signal as additional constraints. We apply the idea of Bregman iterative regularization, first used for image restoration by Osher and colleagues [S.J. Osher, M. Burger, D. Goldfarb, J.J. Xu, and W. Yin, An iterated regularization method for total variation based on image restoration, UCLA CAM Report, 04-13, (2004)]. to recover finer scales. We also present an analytical study of the model disc…
Extropy: Complementary Dual of Entropy
2015
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a complementary dual function which we call "extropy." The entropy and the extropy of a binary distribution are identical. However, the measure bifurcates into a pair of distinct measures for any quantity that is not merely an event indicator. As with entropy, the maximum extropy distribution is also the uniform distribution, and both measures are invariant with respect to permutations of their mass functions. However, they behave quite differently in their assessments…
The on-line curvilinear component analysis (onCCA) for real-time data reduction
2015
Real time pattern recognition applications often deal with high dimensional data, which require a data reduction step which is only performed offline. However, this loses the possibility of adaption to a changing environment. This is also true for other applications different from pattern recognition, like data visualization for input inspection. Only linear projections, like the principal component analysis, can work in real time by using iterative algorithms while all known nonlinear techniques cannot be implemented in such a way and actually always work on the whole database at each epoch. Among these nonlinear tools, the Curvilinear Component Analysis (CCA), which is a non-convex techni…
The Duality of Entropy/Extropy, and Completion of the Kullback Information Complex
2018
The refinement axiom for entropy has been provocative in providing foundations of information theory, recognised as thoughtworthy in the writings of both Shannon and Jaynes. A resolution to their concerns has been provided recently by the discovery that the entropy measure of a probability distribution has a dual measure, a complementary companion designated as &ldquo
SCORING ALTERNATIVE FORECAST DISTRIBUTIONS: COMPLETING THE KULLBACK DISTANCE COMPLEX
2018
We develop two surprising new results regarding the use of proper scoring rules for evaluating the predictive quality of two alternative sequential forecast distributions. Both of the proponents prefer to be awarded a score derived from the other's distribution rather than a score awarded on the basis of their own. A Pareto optimal exchange of their scoring outcomes provides the basis for a comparison of forecast quality that is preferred by both forecasters, and also evades a feature of arbitrariness inherent in using the forecasters' own achieved scores. The well-known Kullback divergence, used as a measure of information, is evaluated via the entropies in the two forecast distributions a…
Coherent Conditional Previsions and Proper Scoring Rules
2012
In this paper we study the relationship between the notion of coherence for conditional prevision assessments on a family of finite conditional random quantities and the notion of admissibility with respect to bounded strictly proper scoring rules. Our work extends recent results given by the last two authors of this paper on the equivalence between coherence and admissibility for conditional probability assessments. In order to prove that admissibility implies coherence a key role is played by the notion of Bregman divergence.
Coherent conditional probabilities and proper scoring rules
2011
In this paper we study the relationship between the notion of coherence for conditional probability assessments on a family of conditional events and the notion of admissibility with respect to scoring rules. By extending a recent result given in literature for unconditional events, we prove, for any given strictly proper scoring rule s, the equivalence between the coherence of a conditional probability assessment and its admissibility with respect to s. In this paper we focus our analysis on the case of continuous bounded scoring rules. In this context a key role is also played by Bregman divergence and by a related theoretical aspect. Finally, we briefly illustrate a possible way of defin…